Approximations to Stochastic Dynamic Programs via Information Relaxation Duality

In the analysis of complex stochastic dynamic programs (DPs), we often seek strong theoretical guarantees on the suboptimality of heuristic policies: a common technique for obtaining such guarantees is perfect information analysis. This approach provides bounds on the performance of an optimal policy by considering a decision maker who has access to the outcomes of all future uncertainties before making decisions, i.e., fully relaxed non-anticipativity constraints. A limitation of this approach is that in many problems perfect information conveys excessive power to the decision maker, which leads to weak bounds. In this paper we leverage the information relaxation duality approach of Brown, Smith, and Sun (2010) to show that by including a penalty that punishes violations of these non-anticipativity constraints, we can derive stronger bounds and analytically characterize the suboptimality of heuristic policies in stochastic dynamic programs that are too difficult to solve. We study three challenging problems: stochastic scheduling on parallel machines, a stochastic knapsack problem, and a stochastic project completion problem. For each problem, we use this approach to derive analytical bounds on the suboptimality gap of a simple policy. In each case, these bounds imply asymptotic optimality of the policy for a particular scaling that renders the problem increasingly difficult to solve. As we discuss, the penalty is crucial for obtaining good bounds, and must be chosen carefully in order to link the bounds to the performance of the policy in question. Finally, for the stochastic knapsack and stochastic project completion problems, we find in numerical examples that this approach performs strikingly well. Subject classifications: Dynamic programming, information relaxation duality, asymptotic optimality, stochastic scheduling, stochastic knapsack problems, stochastic project completion.